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(54) Process for marking a multimedia document, such as an image, by generating a mark 

(57) Method of marking a multi-media document I I orig 

oria by generating a mark from said document and a 
marking image Ij^ q, comprising the following steps: 

- decomposing the initial document Ig^g according to a 
lossy approximation method ; 
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determining a reference Image 1^ ; 

point-by-point subtracting the initial 
document lo^g and the reference image \^^, and 
according to at least one vector representative of 
the image In order to produce a support image l« ; 

formatting the marking image Ijnf o so as to obtain 
an image li^f^ of the same size as the support image I 
by pre-introducing at least one level of redundancy; 

point-by-point combining the marking Image Ijn, ot 
formatted beforehand, with the support image so as 
to generate a mark or a signature which can be, 
according " to the case, either filed, or internally 
reintroduced Into the initial document 



The combining of image li^ and support 
image l« produces a compact-sized mark which can be 
either stored or reintegrated within the document itself. 
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Technical field of the Invention 

^ [0001] The present invention relates to marking a multi-media document, such as a digitized image, and in 
particular to marking a document by generating a mark that can be, according to the case, filed or reintegrated into 
the original document 

Background art 

iO [0002] The security of computerized documents, and particularly of digitized images, implies being able, at one 
time or another, to carry out a marking of these documents in order to distinguish them without ambiguity by means of 
external information, such as identification, authentication or signature data or any other data. ' 
[0003] Two quite distinct approaches can be considered : 

[0004] In a first approach, marking is internally carried out in the sense that the identification message, the 
distinctive mark, is registered in the document itself, for example a digitized image. The techniques known as 
digital watermarking allow the dissimulation of a watermarking tattoo inside an image in order to "sign" it and 
unequivocally identify it. Digital information is inserted into an image, so as to be invisible to the eye. The 
picture is thus signed, for example by its author, the owner of the right on this image, a certifying authority or 
any authority competent to carry out this signature before broadcasting the considered document, for example its 
transmission or its marketing. This approach has to deal with the difficulty of conceiving a tattooing that respects 
a robustness/visibility compromise. Indeed, the added marking information must be sufficiently discrete to remain 
invisible to the eye because debasing the original work quality would be intolerable. However, as discrete as It is. 
the information must be very present in the image so as to ensure its perfect restitution even after the image 
underwent various transformations and processing such as Jpeg compressions, resolution changes, or translations of 
some pixels. In practice, the application of known watermarking techniques is still limited because of the following 
drawbacks. 

25 [0005] A first disadvantage of the known watermarking techniques lies in the low number of bits which digital 
marking information can comprise, the low hiding or the low ratio between the number of bits to hide and the image 
bit volume, or more generally of the document being marked. For instance, for 512x512 pixels images, a coded mark 
with only 64 bits often represents a limit. Obviously this limits the volume of the marking information considered 
to be written in the document. For example, if 64 bits are used to form alphanumeric characters using 8 bits each (a 
byte), then the mark or signature will only comprise 8 characters at most and will thus be deprived of the user- 

30 friendlyness and degrees of freedom from which a user would benefit, could he/she use a much greater number of 
bits. 

[0006] Another disadvantage of the known marking processes lies in the signed images poor robustness, since It 
is often impossible to recover the written mark from the marked image, from the moment the image undergoes a 
number of transformations, whether intentional or not, like translations, image rotations, a cut of some pixels, etc 
... or merely the scanning of the printed considered docum^nt Lastly, known marking techniques also suffer from the 
problem that, to recover the signature or mark, it is generally necessary to have the original document, If not the 
original signature itself. 

[0007] Another disadvantage of known maricing and signature techniques lies in the relatively important visual 
degradation of marked images as soon as the bit size of the mark tends to expand. To mitigate this disadvantage, one 
can be tempted to resort to a second approach in order to respect the original work or document integrity. This 

4Q second approach - known as external approach • consists in generating a reduced format characteristic imprint of 
the original image and marking message to be stored thereafter, for example to a third party. In this approach the 
question of the visibility does not arise any more since the original image can be broadcast directly without 
distortion. Moreover, this approach is highly desirable in certain very sensitive applications for which fraudulent 
attempts to reach within the marking algorithm must be avoided, internal approaches remaining very sensitive to that 
problem. Indeed, with internal approaches, since the marking sign is inside the image itself, a careful observer 

45 might study the image thereby penetrating its secret mechanism. 

[0008] The internal and external approaches are not exclusive of each other but rather complementary. In some 
cases absence of distortion and resistance to the malevolent intrusions which the external approach allows will be 
favoured. In other cases the internal approach wilt be preferred for its easiness of use as it allows to directly 
sign an image that can then be directly broadcast without a certifying person or authority intewening for example. 

^ [0009] Whatever the retained approach, internal or external, it is always desirable to lay out a general image 
processing technique which makes it possible to generate a mark which can be, according to needs, either reinstated 
within the document itself as its "signature" in an internal approach, or used in an external approach, for example 
for storage to a third party. In the latter, it is essential that the mark presents a sufficiently compact format to 
facilitate its storing while inten^^eaving elements of the original document and of the marking information. On the 
contrary in the internal approach, the process will have to make it possible to accurately reconstitute the marking 

55 information when the original image is presented, even after this one undenwent many complex and nondestructive 
transformations and processing. 

[0010] Such is the aim of this invention which proposes to provide a greatly versatile solution, which can be 
used, according to needs, both in an internal approach and in an external approach. 
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[0011] The French patent applications, not yet published, N *FR 98 04 083 (Co pending PCT application 
PCT/FR99/00485), PR 98 07 607, respectiveiy deposited on March 5, 1998 and on June 9, 1998 by the owner of this 
application, constitute technical background within the meaning of article 54(3) of the European Patent Convention 
and describe improvements brought to watermarking techniques. 

5 Summary of the Invention 

[0012] The object of the present invention is to provide a method of marking a computer document, such as an 
image for example, of great versatility since it must make it possible to generate a mark which can be used either 
in an internal way or an external way. 

[0013] It is another object of the present invention to propose a method for generating an external mark which is 
compact enough to allow an easy filing, but also robust in order to allow reconstitution of the marking data even 
after the original image underwent many nondestructive transformations. 

[0014] It is a further object of the present invention to provide a method for generating an internal mark which 
allows to write a significant information volume into a computer document while reducing the distortion brought to 
it, and is sufficientiy robust to allow reconstitution of the marking data even when the document underwent many 
^5 transformations of nondestructive nature. 

[0015] To reach the set goals, the present invention proposes a method of marking a multi-media 
document l^^ig by generating a mark from said document and a marking image o< comprising the following steps: 



20 



decomposing the initial document 1^^ according to a lossy approximation method ; 
determining a reference image l^ef ; 

- point-by-point subtracting the initial document and the reference image l^ef., and according to at least one 
vector representative of the Image in order to produce a support image 1, ; 

25 - formatting the marking image Ijnf o so as obtain an image linn of the same size as the support image 1^ by pre- 
introducing at least one level of redundancy; 

- point-by-point combining the marking image Ij^, q. formatted beforehand, with the support image so as to generate 
a mark or a signature which can be, according to the case, either filed, or Internally reintroduced into the 
initial document Lrin. 

[0016] Thus, the combining of Image I and support image I after thresholding of the latter, by means of a 
logical operation will produce a compact-sized mark which can be stored. On the contrary, "modulating" or varying 
support image 1^ by the value of Ij,^ according to a law of correspondence between 1^ and l^f^, will produce an 
35 internal mark which can be reintroduced in the reference image so as to generate a signed image. 

[0017] More especially, the present Invention proposes a method of marking by generating an external mark. More 
particularly, the method produces an external mark from an original document lo^ig and from a file comprising a 
marking image Ij^f o- The method comprises the following steps: 

- decomposing the initial document l^ng according to a method of fractal analysis while providing a IPS code, for 
instance of contractive type ; 

- determining a reference image l^ef associated vwth said IPS code ; 

- subtracting the initial document \^ and reference image 1,^ , point-by-poInt, and according to at least one vector 
^3 representative of the image, followed by thresholding the resuft to produce a binary support image I, ; 

• formatting the marking file Ij^^ q so as to obtain an image Ij^f^^ whose size is comparable v^h that of the support 
image Ig by pre-introduclng at least one level of redundancy ; ~ 

- combining, point-by-point, the preformatted marking image Ij^ q (i.e. Ij^f i) with the image known as the support 
50 image by means of a logical operation, such as a XOR or an inverse XOR. The result then produces an external 

mark which can be filed. 

[0018] This marking data can be used for whatever purpose It may serve. In particular it can be an 
identification data of the image, a copyright note or a note specifying the kind of Information which is contained 
in the computer document, even a confidential note to automatically locate the document circulating on a network. 
[0019] In another mode of realization, selected IPS code can be of noncontractive type and image l^ef is then 
calculated directiy from the original image l^ng. 
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[0020] In a preferred mode of realization, the invention uses a secret key device allovving introduction of a 
pseudo-random noise in the marking image, before combination with support image t,. 

[0021] Accordingly, the invention proposes a process making it possible to reconstitute, from the filed mark, 
marking data associated with the original document even after the latter underwent some nondestructive 
transformations. More particularly, the process includes a step of decomposing the multimedia document 
long according to a fractal analysis method while providing a IFS code, for example of contractive type. 
Corresponding attractive image IV^f is then generated. Then occurs the point-by-point subtraction of the multi-media 
document l^ria and attractive image \\^, and according to at least one vector representative of the image. This 
operation is then followed by thresholding of the result to produce a binary support image I's • The next step is that of 
combining the mark, disnolsed by means of type XOR or inverse XOR logical operation, with said support image t ^ by 
10 preliminary proceeding to reframing or resynchronization of the latter to take into account any possible 
transformation of the image r^na to analyze. Finally the result of this combination, after reframing, can then be 
formatted to excerpt, if necessary, the marking image contained inside the external mark. 

[0022] In another mode of realization, the fractal decomposition resulting in l^ef can be replaced by other lossy 
approximations, such as the one resulting from a Jpeg coding, a cosine coding or a reduction of the image resolution. 
[0023] In a particular mode of realization, the invention relates to a method of management of owner rights and 
the marking image carries the logo of the owner of the right 

[0024] In a particular application, the marking image is used for control in a network node, in co-operation 
with an adequate sensor device which can then control interception of the original image Igpg or a long form derived 
from it. 

[0025] To reframe a point-by-point combination of the mark, which was disnoised by means of a logic operation, 
2^ with said support image \\ is performed, by an elementary geometric transformation T on said support image l^. An 

iteration intervenes on the preceding combinations for a whole collection of elementary transformations extracted 

from a dictionary, or codebook, and a consistent score is then calculated which is intended to maximize the 

redundancy effect that vyas introduced into the marking image on a local and general level, for each transformation. 

[0026] Alternatively, the invention proposes a method of signing a computer or electronic document by 
25 introducing an internal mark, such as an image, which can be implemented independently of the generation of an 

external mark. In this case, the mark is simply reinstated Into the image itself so as to allow its later 

extraction. The signature process involves the steps of: 

- decomposing the initial document l^^g according to a lossy approximation method; 
30 ' determining a reference image l^^f ; 

- subtracting the initial document 1^^^ and reference image Iref , point-by-point, and according to at least one vector 
representative of the image In order to produce a support image 1^ ; 

- formatting the marking file Ij^f o so as to obtain an image 1,^, ^ whose size is comparable with that of the support 
image Is by pre-introducing at least one level of redundancy; 

- combining, point-by-point, the preformatted marking image Ij^f , vAth the support image Ig in order to produce a 
signature Ij^f 3. 

40 

[0027] More particularly, the ljn,2 signature is incorporated into the l^of image and results from "modulation" of 
Preformatted marking image according to the GAP value contained in support image l„ that is to say between the 
original image or the initial document l^,^ and the reference document and according to law of correspondence 
between GAP and noised marking image The following law of correspondence can be employed: 
'inf 2(^. y) - Inf .(GAP(x, y); 0) or Sup (GAP(x, y), 0) according to whether Ij^f ^ is respectively equal to 0 or to 1 . 

45 [0028] Operators inf and sup respectively represent the mathematical operators providing respectively the 
smallest and the largest values considered. The signature process is such that the pixels which «carrv y> the digital 
mark will tend to qualitatively move away image from 1,^ and, on the contrary, to bring it closer to loria, thus increasing 
the original fidelity of the signed image t'ong towards the original image Io^q. It is an important advantage resulting 
from the signature process. Various alternatives can be introduced on the way of modulating Ij^f 2 between 0 and the 

50 value of GAP, by choosing other terminal values, for example 1 and 1,2 times the value of GAP, instead of 0 and one 
time the value of GAP. 

[0029] Once determined, the internal signature Ijnf 2 can then be added to image 1,^, which makes it possible to 
produce the signed Image which can then be broadcast* 

[0030] Correspondingly, the extraction process of the signature I'in, 2 ^rom a modified image \'^^ comprises the 
steps of: 

55 

- decomposing the document I'^ng to be checked according to the decomposition process that was used for the 
signature process; 



determining a reference image 1'^^ ; 
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- subtracting the document I'ona to be checked and reference image \'f^ , point-by-point, and according to at least 
one vector representative of the image in order to produce a support image l'^, from which the signature rj^, ^ can 

^ be deduced; 

- comparing said signature Tin, 2 with a predetermined threshold value and deriving the value of the Preformatted 
marking image Tj^^ , in accordance with the following formula: 

if l'tnt.2(*^.y) > predetermined threshold, then \)^^ , (x.y) - 1 

10 rf ri„,_2(^.y) threshold then iv^,^^ (x.y) = 0 

- reframing said performatted marking image in order to take into account any possible transformation of the image 
to analyse by means of a search of coherence with the level of redundancy introduced in the signature process in 
order to reconstruct, when existing, the marking image which can then be displayed, A substantial advantage of 
this extraction process is to allow full blind retrieval of the mark since neither the original image nor the 
mark hidden in that image are necessary for the extraction process. 



[0031] The process of generating an external mark and the process of generating an internal mark are completely 
independent. It is thus clear that the invention can be used merely in one of these cases, for example, to produce 
only one signed watermarking «type » image, or conversely. 

Description of the drawings 

[0032] Embodiments of the invention will now be described, by way of examples only, with reference to the 
accompanying drawings, wherein: 
25 [0033] Figure 1 is a schematic representation of the method of generating an external mark in accordance v^th 
the present invention. 

[0034] Figure 2 is a schematic representation of the process of marking data reconstitution from an 
image l^ng which may have undergone various transformations and processing. 

[0035] Figure 3 represents an example of sign to be inserted in an image, such as a signature in the form of a 
logo, as well as its transform after completion of a first preprocessing phase of enlargement; 

[0036] Figure 4 represents the signature example of figure 3, after a second preprocessing step of transform 
duplication. 

[0037] Figure 5 illustrates a particular application of the method of generating an external mark for the 
purpose of constituting a digital image recording protocol in a certification server, for instance on the Internet network. 
[0038] Figure 6 illustrates the signature checking protocol within the certification server. 
35 [0039] Figure 7 illustrates the method of generating a rnark which can be reintroduced in the processed image, so 
as to produce a signed image. 

[0040] Figure 6 illustrates the signature extraction process within the signed image. 

[0041] The figures 9a and 9b are respectively monodimensional diagrams of signals used during rebuilding of the 
signature information, illustrating reframing of the noise and of the signature. 
40 [0042] Figure 10 illustrates the consistency test in the process of extraction of the internal mark. 



Description of die preferred embodiments of the Invention 



[0043] One of the functions of document marking is to, internally according to Xhewatenvarking techniques or 
externally, write digital information which constitutes a marking sign to be used for various reasons. A first use 
which comes immediately to one's mind is an identification sign, such as a signature or a tattoo, which makes it 
possible to relate the computer document to an entity, a person or a company. But obviously other aims can be 
considered, which will depend on the particular function assigned to the marking sign : marking of a document to 
characterize the kind of document, note of confidentiality etc... By means of non restrictive example, the invention 
will be described in relation with use of a computer document signature mark, which mark v^II consist of a company's 

^ logo, for example that of Eurecom Institute, owner of this patent application. 

[0044] Realization of the invention in the case of the marking of a document which contains a fixed image will 
also be described, being understood that the invention is not limited only to fixed images, but can be adapted to 
other kinds of documents, for example digital audio, video etc. The only restraints imposed by this algorithm are 
volume restraints. To give an order of magnitude about binary information volume, in simulation marking signs made 
up of image files containing a logo of 64x64 pixels coded on two binary levels were usually used. That represents an 

55 information volume of several thousand bits which is considered to be associated with 512x512 pixels images, each 
one being 3 bytes coded, where nevertheless only one byte (luminance) is really used to produce the mark. This 
particular, and by no means restrictive, choice which will be described is based on the concern of preserving the 
possibility of marking images in gray levels or shades, or of finding the marking data when the original color image 
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is represented in black and white. 

[0045] We will now consider successively the method for generating an external mark to be filed (I), then the 
adaptation of this method for generating an interna! mark which will be reintroduced in the processed image itself (II). 

^ I. Using the Invention to generate an external mark which can be filed 

[0046] The process of generation of an external mark will be particularly described in reference with figure 1. 
Although the process can be employed with any lossy approximation method, such as Jpeg, a cosine coding or by 
reduction of the resolution, the invention will be particularly described in reference with auto-similarity fractal 
analysis such as described in detail in the application PCT/FR99/00485 and which contents are herein incorporated by 
reference. Original image l^na is processed according to a first step 1 in order to generate a Iterated Functions 
Systems (IFS) code which is characteristic of the original Image This processing is based on auto-similarity 
analysis which results from the fractals theory. Though the auto-similarity concept is essentially used in image 
processing for compression purpose (for example in fractal coding), herein it is described how fractals can be used 
to adequately calculate a reference image l^^f which can be used to generate a mark, externa! in this case, which 
presents a compact format allowing easier storage, and is advantageously filable, for example to a certification 
15 server. But as discussed previously, fractal coding can be replaced by any other lossy coding or approximation. 

[0047] The invention will be particulariy described In reference with image processing in grey shades or levels. 
In the case of auto-similarity fractal coding, auto-similarity implies describing an image either in terms of 
«pixels » associated to a grey shade, but under the form : such area is «similar» to such other area, a given 
transformation modulo. Hereinafter ^ccontext » will designate a selection of a number of primitives for image 
cropping, the way they are of laid out, the matching criterion (that is to say the total "distance" between 
reference image l^ef and initial image l^ria derived from the elementary "distances" between the various areas), as well 
as all authorized transformations between primitives. 

[0048] The initial image to process, called lono. is cropped into a set of primitives, called ranges, which must cover 
the whole image (possibly with an overiapping). Moreover, one basic dictionary Dbase is built which consists 
of domain primitives. The elements of this dictionary are also extracted (by default) from the image itself, but 

25 they do not need to cover the whole image to process. There is no particular constraint regarding distribution of 
the "domain' primitives. The dictionary thus built, is then enriched by using geometrical and photometric 
transformations to form a final dictionary of authorized transformations Dfinal. It should be noted that elements can be 
added to the Dfinal dictionary which is not available to external users. Typically affine type transformations will be 
employed to create new primitives, but other more complex transformations can also be used. Transformations can 
include geometrical operations, such as isometry (i.e. symmetries, rotations, reflections), or photometric 

30 transformations, for example on luminance values. In particular, the context will make it possible to determine 
whether the IFS code has contractivity qualities. It should be noted that to implement the process with the use of 
software, the defined context is declared with the software responsible for executing calculations relating to the 
generation and extraction of the mark. Then each range primitive must be matched to a dictionary domain primitive 
according to a distance criterion, for example the Minimal Mean Square Error (MMSE) which is well-known to people 
qualified in the art. Conversely, in some cases it can be interesting to move the reference image l^ef away from the 
original image 1^^^^. Usually the Euclidean distance will be used, but other distances can be used. 
[0049] While indexing the range primitives to i, the associated transformation will be noted Wj (i.e. the 
transformation which defines an element of the basic dictionary, plus a transformation). Local transformations (W;) 
union will be noted W, the image made up of all range primitives and roughly represented from an element of Dfinal 
will be noted Wl, and the attractor associated with W will be noted A if it exists. It vAW be reminded in this 

4Q respect that, if the code selected is contractive, it is possible, by using the "coilage theorem", to consider attractor A 
(that is to say the image resulting from n reiterations of transformation W) from any initial image \q or : A = W {\^^^), 
instead of transformation Wl, which is the union of range blocks that are approximated from lo^g : Wl is the transform of 1 
oriQ by W transformation applied once: Wj = W (l^^). It is pointed out that a transformation W defines its attractor 
A, since the transformation W applied n times to any image results in image A when the selected IFS code is 
contractive. When the selected IFS code is not attractive, then reference image must be derived directiy from 
original image 1^^^ with formula : l^^f = W {\^^). 

[0050] Illustratively, in an example of implementation of the invention, the following context which can be 
classically used in image compression can be considered. The primitives are square blocks, thus operating a 8x8 
pixels matrix zoning oiranges, and 16x16 pixels matrix zoning for domains. Authorized transformations of the fields 
are : under-sampling by 2; 0 * rotation (that is to say identity), 90* rotation, 180° rotation and 270** rotation; 
^ symmetries regarding geometrical transformations; and a shift and scaling operation of luminance values. 
[0051] In other terms, each local transformation Wj can be written as : 
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where a;, b„ c,, dj represent the geometrical transformation (over/sub sampling, plus rotation/reflection,...) ; 
(x,y) represent the image point coordinates and z represents the associated gray shade ; (Uj, Vj) represent the 
displacement which is necessary to match the domain primitive position with the correspondingrange primitive ; 
Sj and Oj respectively represent scaling and shift of the values of luminance. 

^ [0052] Thus, at step 1 in figure 1, the processing aims at describing the image Igna by means of IFS code. To this 
end, as mentioned above, the original document Iq^^ is cropped into a set of primitives range(i). which cover the 
totality of the original document. Then, a set of primitives domain(i) is extracted in order to constitute a dictionary 
of basic primitives domainO'} (Dbase). From the dictionary of basic primitives domain (Dbase), is determined a 
dictionary Dfinai of additionaldo/rra/r? primitives, by applying a set of geometric and/or photometric local 
transformations wi to basic primitives. In certain modes of realization, it is even possible to introduce secret 

10 elements into the Dfinai dictionary in order to reinforce the security of the process of external mark generation. 
Thereafter, each basic primitive range(i) could be paired v^ a DomainQ) primitive included in the primitive 
dictionary Dfinai, in such a way that each primitive ranga(i) is equal to a primitive domain(j) transform obtained by a 
local transformation Wj, to within about one error e. The image description thus obtained is rather compact, even if the 
compression effect is not really seeked. This description is a IPS code that can be concretely gathered in a file in 
a form which can vary according to the selected mode of realization. It should be noted that the IFS code is very 

15 dependent on the determined context, that is to say on the cutting into primitives which is selected, as well as of 
the chosen set of affine transformations. This context is particularly selected according to the particular 
application which will be selected for the process of external mark generation. In particular, it should be noted that, 
to ensure the marking image reconstitution process robustness, it is necessary that the geometric transformation 
dictionary be selected to correspond to elementary transformations which the image might undergo and which, during 
the marking image extraction process, will allow to then reconstitute the logo. 

20 [0053] The choice of the dictionary, that is to say the context, enables to ajust qualitative differences 
between image 1^^,. and image lopQ. Moreover, when the selected IFS code is not contractive, then, as dicussed before, 
image l^ef. directiy results from original image 1^^ by formula U = W (l^^^ ). On the other hand, when the selected 
context is contractive, then attractor image is then built from any starting image lo or Ij^it - black, for example by 
means of a series of images W"^(lo) which will converge towards a limiting value which Is the attractor image 1^^. This 

25 property comes from the fractal attractor theory, resulting from XheMandelbroVs fractal mathematical theory 
and hiutchinson's systems of reiterated functions, such as applied to Bamstey's image processing. Whatever the 
chosen code, attractile or not, step 2 of the process illustrated in figure 1 thus makes it possible to constitute 
this \^ image which is necessary to the generation of the support image 1^. 

[0054] In step 3, image and \Qf^ image are then combined in order to generatefhe support image \^ intended to 
be combined with the marking image. To this end a point-by-point subtraction of vectors representative of the image 
is performed. It should be noted that various compositions of vectors are possible according to circumstances. In 
particular the vector can be a primary colors vector, red-green-blue, or can include a luminance characteristic 
component - allov^ng gray shade processing - even other bands for multispectral image processing produced by 
satellite imaging. In order to simplify this discussion and to clearly understand the various steps of the operating mode 
of the invention, only processing of the luminance characteristics components will be discussed, that is to say in 

35 gray shades. It is also possible to reproduce the vectorial phases of the process for other components. In step 3 of 
the support image generation, a point-by-point subtraction of luminance components is carried out A difference 
image with numbers characteristic of gray shade offsets is then obtained. This image is then subjected to a first 
robustness processing by means of a thresholding operation, to define the offset direction, positive or negative. 
This operation ensures a first level of robustness of the algorithm, for only the sign of the offset is taken into 
account and not its exact amplitude. Following this processing, a binary image ready to be combined in order to 

40 generate an external mark is obtained. 

[0055] Simultaneously to stages 1-2 and 3, the process carries out a processing of marking image ti^f q which, as 
mentioned before, is an image which size is significantiy lower than the size of image This processing is carried 
out at steps 4 and 5 which are illustrated in figure 1. 

[0056] Figure 3 presents an example of binary information 10, a marking image containing the 
distinguishing 'sign v&\\ch vM be associated v^h original computer document Iqhq. In the preferred mode of realization, 
the sign is made up of a fixed image which can present all kinds of forms according to the industrial applications 
considered. The fixed marking image could thus consist of plain text, but also form a company's logo, as illustrated 
in figure 3 which shows as an example the logo of the "Institut Eurecom," owner of this patent application. This 
marking image will be noted: Ij^f q. 

[0057] Step 4 then ensures formatting of marking image Ij^f o- ^^'^ ^^d, the latter undergoes a processing 
50 intended ta bring back its size to that of image 1^^^ for one part, and to introduce a redundancy effect which increases 
the robustness of the process of marking image reconstitution . A double redundancy effect is inserted : a 
redundancy at local level, followed by another at global level. 

[0058] On a local level, redundancy is made by means of an oversampling of marking image Ij^f q, consisted by 
Eurecom's logo. This oversampling is carried out according to vertical and horizontal directions in the manner of an 
enlargement or zooming. To this end, the binary pattern represented by marking image 10 is enlarged, so that each 
pixel of Ijnf 0 is k times in an intermediate transform 20, noted Ij^ o -i- "''hus. figure 3, Eurecom's logo enlarged in 
this intermediate transform Ij^^ resulting from the initial logo which was smaller, is represented. Each pixel of 
primary marking image ljnf_o was duplicated k times in a vertical direction and j times in a horizontal direction, so 
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that a basic pixel of the initial logo Ij^ q is now represented by k x j Identical pixels which are regrouped. Thus, the 
same pixel is duplicated several times at a local level. This allows introduction of some replication effect, which 
will allow error correction and, if necessary, synchronization which will be necessary during the reconstitution of 
the marking image as we will see further. 

[0059] It was determined, for low-pass filterings not degrading the marking image-sign quality too much, that a 
k = j = 3 oversampling factor constituted a suitable parameter in general. This oversampling involves an offset 
towards the low frequencies of the original logo, since the level of a bit is maintained with the same value 
throughout all oversampling. This frequential offset is consistent with the idea that low frequencies will slightiy 
be deteriorated if the image preserves an acceptable quality and, conversely, that high frequencies are likely to 
disappear during a low-pass filtering not degrading too much the primary marking image -sign, which is classically 
admitted in the field of lossy coding. 

[0060] A second level of redundancy is then introduced on a global level at step 4, which lies on a duplication 
of the intermediate transform Ij^f q ^, that is to say the marking image after local level enlargement of the logo. A 
secondary property formatted marking image Ij^f 02 thus appears, which is illustrated as a paving such as in figure 4, 
after enlargement on a local level and pattern duplication on a global level. In a particular mode of realization 
(which does not correspond to figure 4) the formatted marking image tj,,, 0.2 generated by means of an oversampled 
^5 duplication of marking image Ijn, 0.1, thirteen times in a vertical direction and four times in a horizontal direction. 
But other values are possible. In particular, the number of duplications to perform to form secondary marking Image I 
in, 0 2 can be selected so that the latter has the original size of the original image l^ria to be marked. In the general case 
where the duplication of the primary marking image-sign to be written into the external mark, that is to say the 
binary message, takes place L times, each pixel of primary marking image (the number of these pixels being with n) 
thus is k X I times in redundant secondary marking image Ij^, 0.2- 

[0061] The second level of redundancy by duplication is recommended to mitigate an algorithm failure on the 
local level. It is indeed essential to be protected against any effect resulting from a loss of information on a 
local level - in a vicinity of some pixels - due to a particular processing of the image. During processing, 
such as cropping, areas of the original image disappear and it would thus be extremely prejudicial that all 
information enabling reconstitution of the marking image - i.e. the logo of "Institut Eurecom" - be contained in a 

25 single area of the image. This second duplication which is carried out on a global level this time, by reproduction 
of the enlarged logo, still increases the possibility of synchronization and error correction which the step of 
pixel enlargement on a local level already facilitated. This double processing of the marking image, by local 
enlarging and global duplication of the pattern, is perfectiy appropriate for processing a fixed image but could 
easily be adapted to the processing of files of different nature, and in particular of the video files. The double 
effect of redundancy which results from formatting step 4 adds an additional element of robustness to that which 

30 already rose from the thresholding operation which was carried out in step 3 of calculation of the support image. More 
particulariy, the balance of the components of the processing of step 4 which is described above, namely logo 
enlargement and duplication, depends on the objectives of robustness desired for the process. The final development, 
in particular the determination of values k and L can perfectiy be performed by anyone qualified in image digital 
processing. Thus, in the case of video files, lower k and I values will be be satisfactory - and more generally 
lower redundancy will be satisfactory for format processing of the image. Indeed, because of the volume of 
information to be processed - I.e. several tens of fixed images per second - a redundancy effect that would be 
reduced concerning the format processing of the logo would be found elsewhere. 

[0062] After the adding of redundancy, at a local level as well as a global level, a secondary marking 
image Ij^, 0.2 which is perfectiy adapted to the size of image long to be marked and is also binary is provided. As 
mentioned, the step of formatting the marking Image brings a factor of robustness which adds to the robustness 
40 effect already observed in step 3 during generation of the support image. 

[0063] After step 4, the process of generating a mark can optionally comprise a noise element In a step 5. This 
step 5 depends on the desired level of security and of the computing power which Is available. If it is wished that 
any person can regenerate the logo, during the process of reconstitution of the marking Image, without having to 
resort to any key, then step 5 can be omitted. On the other hand, it can appear extremely convenient, in order to 
increase the .security of the device, to give the possibility of reconstituting the marking image, i.e. the logo, 
only to a small number of people, or even a single Individual. 

[0064] To this end, a random noise of the same size as secondary marking Image Ij^, 0.2 's generated. Then, the 
noise is introduced into the secondary marking Image by means of an EXCLUSIVE OR operation, noted XOR, between 
secondary marking image li^f 02 on one hand and the random noise signal, noted B, on the other hand. This random 
noise can be generated by an algorithm with a secret key K,. In a particular mode of realization, the key Kg could be 

5^ constituted, of two parts: a first part, known as public part, which altov^ relatively large access to one part of the 
message, for example to reveal a copyright note, supplemented by a secono part which v^It remain more confidential 
and which, for example, will be reserved to the owner of the right or certification authority for Instance. In 
another mode of realization, the key could Include dating data of the external enabling in particular to securely 
establish the moment of creation of the computer document 1^. It should be noted that this random noise Inserting 
step is optional and depends upon the requirements to enhance security In the marking processing. Obviously, If the 

55 filing system which Is intented to receive the filed marks provides sufficient security vynth respect to external 
intrusion, the optional step of noising can be avoided. 

[0065] The result of the processing of step 5 leads to a noisy marking image Ij^f ^ which is deprived of 
repetitivlty. The presence of the noise in this image U 1 has a double advantage. On one hand it suppresses any 
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periodicity which could result from the double redundancy introduced at step 4 and, on the other hand, the noise 
allows to make the distribution of the marking image that must be keyed Into the external mark uniform. Indeed, by 
means of this noise B, information Ij^ is made perfectly random and statistically independent of marking image Ij^, o- 
In practice, a uniform noise can be used on inten/a! [ 0 ; 1 / oversampling penoo]. The maximum frequency is imposed 
by the period which is adopted during oversampling of the primary image-sign, that is to say the initial logo. 
^ Indeed, it is known that it is prejudicial to introduce a frequency signal higher than 1/period of oversampling, for 
this signal would be very sensitive to low-pass filtering of the image. More generally, the choice of the noise is a 
matter of optimisation understandable to people qualified in the art 

[0066] The process of generation of an external mark continues then with step 6 during which support 
image \^ and secondary marking image 1;^ q.j formatted and possibly noised according to step 5, are combined to 
iO produce an external mark according to the invention. This combination is done by means of an XOR operation, possibly 
reversed. For example, at each pixel level, a comparison is made and, if bits appear equal, the external mark will 
be allocated a logic "1" and, in the other case, a "0". A binary image Is thus obtained, which is a hybrid result, 
composed of image original 1^^^ and marking image Ij^ q. which can then advantageously be filed to a database, for 
example to a certification server, or any other third party.' 

[0067] A first advantage of the invention is visible, which comes from the fact that, the external mark being 
binary, a substantial reduction of storage space within the certification server is obtained. Moreover, in this case 
original image i^j^^ remains intact and can be marketed and made public just as it is, v^out any restriction. 
[0068] But undoubtedly, the most important advantage lies in the possibility of being able to automatically find, 
within the certification server, an association existing between two images l^^j^ and Yj^^, where one is derived from the 
other, through to the process of reconstitution which v\rtll be described hereafter A first suspect image, which one 

20 has some reason to think is deriveo from an original image, can thus be analyzed with this original image, even when 
only a reduced mark of this image is available. A considerable lightening of the certification server storage load 
results, which even allows to consider filing a great number of Image marks within a single database. As it is seen. 
In the process described, all pixels carry image, which increases the robustness of the message extraction. It is a 
significant advantage of the external approach because on one hand, all the pixels of the mark are used to register 
elements of the marking image-sign, that is to say the binary message and , on the other hand, redundancy within the 

25 external mark can be more easily increased. 

[0069] The process of checking makes it possible to test an image t'ona which is potentially derived from an 
original image through a certain number of elementary geometric or photometric transformations, and to 
automatically relate this image to a mark stored in a database and to return, in the event of matching, the logo 
associated with the mark. The process of checking thus makes it possible to test an unknown image, and to relate it 
to a mark stored iri a data base, to finally automatically extract the message or logo associated therewith. 
[0070] Image I'ona to be tested could be a new image, in which case no message would be reported since it would 
be entirely original. But image I'orjo could also be derived from a prior image lona whose mark will have been filed in the 
certification sen/er data base. When at a given moment, the owner of the right on the original image 1^,^^, or any 
interested third party, is in presence of an image l'^^^, this one will seek to check if this y^^^ image results from a prior 
original image. It v\^ll thus be necessary to check image to see if it is possible relate it to a stored mark and, 

35 if necessary, to extract therefrom the logo which is contained in marking image Ijn, q. 

[0071] The process of image checking and of marking image extracton will be more particularty described in 
relation to figure 2. 

[0072] At step 11, image to be checked is submitted to IFS code calculation, which is similar to the 
calculation which governed the process of external mark generation in step 1 of the process of generation of figure 
1. To this end, the same set of primitives and the same dictionary of affine functions are usually used, although it 
is not absolutely necessary. People qualified in the art could indeed use some dissymmetries between the two 
dictionaries in order to optimize the marking, retrieval and recognition system . 

(0073] Then, in step 12, the process of checking carries out a rebuilding oiattractor Image from IFS code. 
[0074] In step 13, attractor image 1'^ is then pixel-by-pixel subtracted from the image to check according to the 
vectors repres.entative of the image, for example the components of luminance, to generate a support image 1^ which 
45 is then thresholded to know the sign of the difference. This support image is then binary and has the same size as 
basic image I'orjg- 

[0075] When the filed marks are noised and encoded according to step 5 of the process of external mark 
generation of figure 1 , then the image checking process is correspondingly associated to a disnoisying operation of 
carried out in steps 14 and 15. To this end, key is introduced to generate the pseudorandom sequence which was 
used during the noisying of secondary marking image Ij^f 0.2- 

[0076] After the generation of the pseudorandom sequence, in step 14, the process then causes extraction of the 
noise which was contained in the filed mark. To this end, in step 15. each bit of this pseudorandom sequence is 
associated v^h a corresponding pixel of the noised filed mark, and submitted to a logical operation, XOR for 
example. It should be noted that the same association law between the image pixels and the pseudorandom sequence 
bits is used as for the step of generation of the mark. For example, the law could be an image scanning law, from 
5S left to right, and from top to bottom, as the law known in television signal scanning. But other possibilities can 
be considered. 

[0077] After the processing of step 15, a synchronization step is necessary before rebuilding of the original logo 1 
jnf_o which was used to generate the external mark. Indeed, due to intentional or unintentional processing the 
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modified image might have undergone, the result of the support image could be a translated, cut version of 
original support image I, v\/hich was used to generate the mark. In the particular case where the image would not have 
been modified, or when fractal coding alone would absorb the transformations undergone by the original image, both 
support images would be identical, so that the binary message of marking image-sign would be extracted immediately 
without difficulty. This is particularty the case with an uniform increase in luminosity on all the image surface. 
^ [0078] Globally, in order to automatically rebuild the initial logo 1;^, q. it is necessary to operate an inversion of 
the process of formatting of the logo. This involves a XOR operation, for instance, between the disnoised external 
mark and thesupport image \\, The result - which can be referred to as I'jn, 0.2 * ts then analysed by means of 
estimators using as well as possible the information redundancy in spite of the disturbations potentially undergone 
by the image. In other terms, it is possible to relatively readjust disnoised mark (resulting from step 15) with 
support image \\ in such manner that after a series of simple transformations of this support image calculated from 
analyzed image \'^^, a logo which is close to the initial logo Ij^ 0 's found, then one vaW know that the modified image 1* 
oria will have been obtained from an original image 1^^^, and by the same occasion one will have found the signature of 
the owner of the image, or the relevant information associated therewith, such as a confidentiality note. 
[0079] Step 16 has as an aim, on one hand synchronization of support Image, then on the other hand, its 
combination with the mark generated at step 15. Each elementary processing implies using a function T selected from 
a set of geometric functions carrying out modifications of the support image positioning. Functions T such as 
translations, rotations, homotheties, scale changes can thus be successively employed. 

[0080] For example, in a first elementary processing, support image is submitted to a first 2 pixels translation 
to the right. The resuft is then combined, pixel by pixel, with the disnoised mark which v/as generated in step 5. At 
the end of this combination, a test is carried out on the result in order to measure a probability score which 
2Q measures the level of redundancy that the result presents : for example the number of elementary patterns made of 
same value pixels, such as packets of 4 bitsx4bits pixels with 32 pixels interval for example, repeated every 32 
pixels for example. The measure of the recognition index strongly depends on the enlargement and duplication process 
which was used for the message or logo formatting in the process of generation of the mark. 

[0081] Then, in a secona elementary processing, support image is submitted to a second geometric function, 
introducing for example a translation of 2 additional pixels towards the right, and the recognition index 
25 corresponding to this new function T then is generated and compared with the previous index. 

[0082] All T functions resulting in pixel displacement to the right can thus be submitted to similar processing, 
then those resulting in a displacement towards the left ... 

[0083] The other geometrical functions, such as rotations, homotheties and symmetries can thus be submitted to 
similar processing and result in a measure of the recognition index, characteristic of a message perception or a 
logo formatted in the support image which is tested. 

[0084] When all T functions pertaining to the dictionary are exhausted, the process calls upon a decision 
element which will determine if the value of the highest generated index corresponds to a sufficient probability of 
recognition of a filed mark. In general, a threshold mechanism can be used to take such a decision. 
[0085] Thus the successive comparison of all the indices makes it possible to provide a measure representative 
of the matching probability between the filed mark and the in^age to test 
35 [0086] In concrete terms, step16 first starts with reframing the disnoised mark with support image resulting 
from the processing of analyzed image l^f^ in order to ensure combination logical operation of the two entities, in 
particular a XOR. The technique of reframing is based on a search for coherence between the various bits 
contributing to the rebuilding of a pixel of the logo of marking image Ij^ o- To this end, one performs the maximization 
of a criterion which is based on two subcriteria in relation v^h the two levels of processing used in the 
formatting operation step 4. namely the steps of local enlargement of a k factor and of I times global replication 
of the intermediate transform Ij^f containing the enlarged logo. 

[0087] The first subcriterion, known as local, is related to the oversampling of the logo and result in checking 
that the bit value remains constant over the period of oversampling. This condition is met only if the disnoised 
mark has been correctly reframed to support image 1*3 resulting from the analysis of image 1*0^0 to test, with XOR 
operation in step 16. Indeed, before noisying of secondary marking image Ij^, 0 2. the value of one bit of image 1;^^ , is 
locally reproduced k times around this bit, and the value of this bit is still reproduced I times at the global 
level, because of the replication operation which is introduced on the intermediate transform Ij^, 0.1 to produce secondary 
marking image Ij^ 0.2- 

[0088] Consequentiy, the same type of local and total consistency must be found on the result of logic operation 
XOR of step 16. One just needs to test several possible transformations of \\, which correspond to inverse processings 
^ of those Image loria might have undergone before becoming Y^j^^, and several types of relative reframing before XOR 
logical operation. According to the results of these operations, it will thus be known if image V^^^ can be associated with 
a filed mark and, in this case, automatically extract the initial marking image I,,,, q. 

[0089] Consequentiy, reframing of the Ig images and of disnoised mark is done by a series of iterations by which 
successive transformations are applied to I'g and, for each of them its effect on the XOR result coherence between 
support image on one hand, and the filed mark free from noise on the other hand, Is examined. As previously 
55 mentioned, when the image Y^^ to be tested and the filed mark are linked by the same original image long, the 
result of step 16 is a secondary marking image Tin, 0.2 ^^'^^ image of secondary marking Ij^, 0.2 which 

was used to create the external mark. When the offset is correct, secondary marking image l^^f 0.2 produced by step 16 
presents a strong degree of redundancy, both on the local level and global level, that can.be used as a decision 
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criterion. 

[0090] In practice, in order to test the coherence of the result produced by step 16, that is to say a secondary 
marking image Ij^, 0.2 produced by step 16, a function known as coherence function is built on the basis of two 
subcriterion at the local and global level. A score of coherence to be maximized is calculated in the following way. 
^ For each basic binary bit of the logo, In all its occurrences in 1;^^ 0.2. difference in absolute value between the 
pixels whose binary value Is zero and those whose binary value is one is calculated, then all n pixels of the basic logo I 
inf 0 are added up, this sum constituting a score of coherence to maximize by use of possible transformations applied 
to support image I',. 

(0091] Reframing is then similar to the maximization of this function in relation to the transformations 
variable which takes its values In a predefined set : offset, scaling factor, rotation... In practice, it is like 

10 iteratively testing the value of the function of coherence for a whole series of T transformations of the image and 
retaining the transformation or the combination of transformations which maximizes the function of coherence. It 
should be noted that the calculation of each score is independent of the others, and all calculations can be made in 
parallel. In other terms, all transformations T to be tested, to each of which will correspond a score, could be 
realized in parallel. Moreover, the occurrences created by enlarging of the logo (for example) can be more (or less) 
trusted than those created by duplication, or reciprocally. By default, they can not be distinguished. In addition. 

^5 in the present implementation, the set of transformations consists of all the operations of translation of the image. 
ofmirroring effect, of positive and negative zooming, or of combinations. The method could be immediately extended to 
other transformations, if robustness requirements justified it in regard to additional transformations, thus 
authorizing a greater compensation for processing of the marked image. 

[0092] When synchronization is complete, the q logo can then be extracted so as to finally allow visual 
2^ comparison with the initial logo Ijnf q. This construction consists in reversing redundancy addition which characterized 
step 4 of formatting the initial logo. With this intention, a decision module based on a vote of the various pixels of 
secondary marking image \\nf 0.2 's used : the k x I pixels whichvofo in a certain way are replaced with a single pixel, 
in order to pass from I'jnf 0.2 to ''inf o- For a pixel of the given logo, among the bits which are considered valid, a vote 
is organized which is possibly balanced by information which results from reframing, about the occurrences value (0, 
1, unspecified because of the nonsignificant character of the measured amplitude) among all the occurrences of this 
25 element in secondary marking image. Once reframing is complete, the images areas which can be highly trusted can 
then be distinguished from the others, and a weighting which is taken into account at the time of the vote can be 
defined, if the obtained score is higher than a certain predetermined threshold, that is to say If a sufficientiy 
vast majority is gained, the logo pixel takes the value of the main group of pixels. It should be noted that the 
threshold to which the score must be compared is a development understandable to people qualified in the art. 
[0093] At the end of the voting step, a binary image 1';^, q is obtained and its visual examination makes it possible 
30 to determine immediately if it identifies with a logo. Indeed, if 1*^,^ image to be analyzed is an image which is obtained 
from transformations of the 1^^^ image which is associated v^ the external mark stored, then I'l^f q is identical 
to Ijnf 0 within a few bits. In case a visual logo is used, certain erroneous points can be corrected with a vicinity 
coherence criterion : a single white point cannot be isolated in an entirely black area. 

[0094] Thus, in concrete terms, if the marking Image which is built in step 17 appears in the form of a visible 
35 logo - for example the "Institut Eurecom'"s logo -, it means' that marking Image I'jn, □ 's very close to original marking 
image Ij^, q used to build the external mark which is filed in the data base, and which was used to analyze I'o^jg image . 
This then enables a human operator to very simply and immediately note that modified image Tona is only a transform 
of an original image whose mark was stored in the data base of the authentication server. If the analyzed image 
cannot be associated vwth any filed mark, it Is then clear that nothing coherent will be extracted from the image to 
analyze which will then be a completely original image. 

[0095] The invention thus enables to analyze any image and to determine if it is associated with a mark which is 
filed in a database, without necessarily having a copy of the original image which was used to produce the 
considered mark. According to a mode of realization of the invention, the image to be analyzed is iteratively 
compared v^th a whole series of marks stored in the database, so as to almost automatically produce the logo of the 
company or individual who created it or, more generally, marking information to which the image can be associated. 

^5 As a particular application of the Invention, the possibility of automatically testing computer documents which pass 
through a network can be considered, to determine and intercept certain types of documents if needed. 
[0096] As mentioned, the invention allows use of marking images or information of relatively great size, while 
ensuring a good robustness against transformations of the image. Indeed, because of the degree of robustness of the 
process, a relatively restricted dictionary of basic functions T can be appropriate. Thus a reduced set of 
translations and rotations can allow synchronization and reconstitution of the logo or message. Thus, for example, it 

50 will not be necessary to consider all translations, pixel by pixel, to allow resynchronization of the image. On the 
contrary, a limited set of translations, presenting a step a few pixels wide, could be sufficient. 

[0097] Figure 5 illustrates application of the invention in the constitution of a protocol of transaction on the 
Internet network for example, which allows recording of the owner of the right's identification data in an automated 
sen/er. The protocol is composed of an initialization phase and a phase of recording of the images subjected by the 
applicant. It should be noted that this transaction intervenes in general on a teleprocessing network and implies a 
certain number of messages which must be exchanged and whose development is understandable to people qualified 
in the art 

[0098] The first initialization phase starts with step 21 during which the owner of the right on one or more 
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recording images requests recording on a telecommunications network. The certification sen/er answers this request 
by sending an identification request message in step 22. Then, at step 23. the owner of the right sends his 
identification data and a password permitting the server to grant the recording request. Upon reception of the 
applicants identification data, the sen/er sends an acknowledgement of delivery confirming the beginning of the 
image recording session at step 24. 
^ [0099] The second phase then allows successive recording of alt the images subjected by the applicant. In step 
25, the applicant sends on the network the first image which is then analyzed according to the process of the 
invention. The mark can then be generated, and the certification server can then return, at step 26. the mark 
Identification number corresponding to this first image. At step 27, the sen/er invites the applicant to produce, if 
necessary, other images to record and each image automatically processed which allows the mark identification number 
to be sent back to the applicant. When the last image was treated, the certification server then returns an end of 
session message which enables closing of the transaction, at step 28. 

[0100] Figure 6 now illustrates the protocol of automatic checking of image marking data to a certification 
server accessible through a communication network. In this protocol, a first step 31 enables a third party which is 
interested by an image to submit it as well as the mark identification data to the server. In a particular mode of 
realization, the identification data of the mark can appear in a image heading so as to allow any interested person 

15 to check the existence of right on the image. Let us note that falsification of these identification data is not 
advantageous for a malevolent person. In step 32, the logo of the owner of the right is returned by the 
certification server after the implementation of the image analysis algorithm according to the invention. 
[0101] Generally, in such an application of rights management, both processes of mark generation and image 
checking are carried out in a theoricalty protected space, safe from inquisitive eyes. In this manner, the mark 
which is generated is at no time made public, which is an advantage compared to the watermarking traditional 
approaches. Indeed "selected plam text" type attempts to penetrate the algorithm code are thus made difficult. 
Indeed, anyone external to the certification sen/er has no way to obsen/e means the relations of dependence between 
an image and its mark ; thus the person cannot subject a set of images and obsen/e the modifications generated on 
the marks in order to try to penetrate the mark generation algorithm. In the mode of realization which is described, 
the only information which is returned by the certification sen/er after an image analysis is the logo of the owner 

25 of the right. But correspondence between the mark modification and the extracted logo modification is absolutely not 
common because of the strong redundancy mechanisms which were previously described and which are implied in this 
relation. Many modifications of the mark presen/e the logo intact. In a mode of realization, the certification 
sen/er can be programmed so as to return the logo as stored in the data base only when the mechanisms of coherence 
search show its existence on a large part. In the other case, a mere chart can be returned to indicate that the 
considered document has not been recorded. In this situation, the relationship mark modification / logo modification 

20 thus disappears completely for a person remaining external to the certification sen/er, thus accentuating the 
algorithm secrecy efficiency. 

[0102] The process of generation of an extemal mark guarantees the possibility for owners of rights to market 
the images as they are, without loss of quality, while providing the possibility - by remote data transmission • of 
showing a proof of the existence of rights on this image by means of the checking protocol according to the 
invention. When a third person is interested by a particular image, he/she can directiy access, thanks to the means 

35 of the invention, the logo of the owner of the right in a 'reliable and effective manner, such as described above. 
Moreover generation of "an external mark has a double advantage concerning image recording. First for the owner of 
the right who needs only to keep the original images. The certification sen/er or operator which stores the mark 
only has to store a mark which presents a compact and reduced format compared to the original document. If one 
considers for example the processing of a 24 bits color image by the certification sen/er, it comes to filing a 
binary mark, and possibly a secret key, which represents 24 timesless information than the original image. In a 
particular mode of realization, the amount of stored information can be further reduced by means of a lossless 
coding adapted to the binary images. Such a coding is known by people skilled in the art and will therefore not be 
described hereinafter. To implement the process of image checking according to the invention, it Is sufficient to 
use this mark without necessarily having neither the original image, nor of the signature itself. The invention is 
thus well adapted to blind signature extraction. 

^5 [0103] Both discussed protocols are adapted to the management of the identification data of the owners of right 
on the images. But the process of generation of a mark of the invention can be used in other Industrial 
applications. In particular, the process could be used in order to control of data fiow of a company, which could 
register marking data on the sensible images likely to circulate on its internal network. With each input and output 
point of the company's network, an automatic detection device could employ the image analysis process of the 
invention to ensure a controlled diffusion of the images outside its network. 

50 [0104] 'Thereafter, it will now be described how the invention can be adapted in the internal approach to provide, 
on one hand, a process of signature of a computer document such as a digitized image, and, on the other hand, an 
automatic process of extraction of this signature. Like previously, the signature could be made up of a logo as that 
which is represented on figure 1. Of course, it will be possible to use the process of signature and of retrieval of 
this signature without resorting to the external generation of a mark, as was previously described. Instead of being 
stored to a third party, the mark will be simply reintroduced into the original document itself. In this case, it Is 

55 then proposed a greatiy effective process of signature since a signature formed by a whole logo can be written in 
the document, that is to say several thousand bits. 

II Use of the invention for the digital signature of a document 
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[01051 Figure 7 illustrates the process of signature of computer document. The process consists. of a first 
processing of the original Image, by means of steps 101, 102 which are respectively similar to steps 1 and 2 of 
figure 1. Thus, like previously for the process of generation of an external mark, in the particular case of a 
fractal coding, the original image lono 'S processed in order to generate a IPS code which is characteristic of the 
^ original image, in step 101. Step 102 then consists in calculating, in a similar manner to that described previously, 
an image of reference I,b,. Naturally, as for the process of external mark generation, another lossy approximation 
coding different from fractal coding could be used. But in this case, rt should be noted that the choice of the 
context is particularly determining in the process of signature : the not very faithful image of reference which is 
generated by the IFS code can be selected just as it is. which makes it possible to fill the difference existing 
with the original image, to incorporate therein information elements of the mari< to be written. The invention 
particularty uses this difference which can be adapted according to needs and to particular applications from the 
selected context and IFS code. This is a significant advantage which appears particularty important in the internal 
approach since a great amount of information can thus be Incorporated in an image which quality, after processing. 
v\^ll remain close to original image quality. 

[0106] In step 103. the images Ir^f and lono are then combined in order to generate a support image I,. To this end, 
15 a point-by-poInt subtraction of vectors representative of luminance component is performed, by taking again an 
example similar to that described in the process of external mark generation. A difference image with numbers 
characteristic of gray shades variations is then obtained. But contrary to the process of external mark generation 
of figure 1, support image remains free from any thresholding, so that it is not binary. 

[0107] Parallel to stages 101, 102 and 103, the process of signature executes a processing of marking 
image l^f q similar to that described in the external approach, that is to say in reference v^h steps 4 and 5 of figure 1 . 
[0108] If one considers again the example of a company logo, as that of this applicant, marking image of Ijn, o is 
submitted to a formatting step 104 which again enables to bring back its size to that of image 1^^. but also to 
introduce the desired redundancy effects. As previously, a double effect of redundancy is inserted : a redundancy at 
the local level, followed by another on general level. 

[0109] At the local level, the redundancy, is again carried out by means of an oversampling of marking 
25 image Ij^f q, constituted by Eurecom's logo. Each pixel of primary marking image Ij^f o is thus duplicated k times in a 
vertical direction and j times in a horizontal direction, such that a basic pixel of the initial logo Ij^f q is now represented by 
k X j identical pixels which are regrouped. 

[0110] At the global level, a second level of redundancy is also introduced at step 104, which is based on a 
total duplication of the intermediate transform li„, o.^. that is to say the marking image after enlargement at the local 
30 level of the logo. Thus appears a secondary marking image Ij^, q which is property formatted, and which presents 
the form of a paving as that which is illustrated in figure 4. Th© redundancy which is introduced on a total level 
this time, by duplication of the enlarged logo, increases the efficiency of synchronization provided by duplication 
at the local level. 

[0111] After the adding of redundancy, at a local level as well as a global level, a secondary marking 
image linf_o.2 which is perfectly adapted to the size of image Ip^g to be marked and is also binary, is provided. 
^5 [0112] After step 104, the process of mark generation can optionally comprise, as for the process of external 
mark generation previously described, a noise element (sound effect) in an optional step 105. This noise element is 
introduced as previously described, that is to say by means of a random noise which can be generated by a secret key 
K3 algorithm. It should be noticed that, in the perspective of designing a watermarking technique which involves the 
creation and dissemination within the public of the image being marked, the noising step 104 may appear much more 
appropriate and necessary than v^ respect to the external approach. 

[0113] The result of the processing of step 105 is a marking image , , which is deprived of repetitivity when 
it is noised, and that can be used to produce a mark Ij^f 2 which could be inserted into the document to sign and more 
precisely the 1^^ document. Therefore, it appears that the noising step brings the double advantage or increasing the 
security of the marked image and further attenuates the visual effect of the marking process. In the preferred 
embodiment of the invention, this Ij^f 2 mark results from "modulation" or variation of noised marking 
image Ij^ ^ according to the GAP value contained in support image l„ that is to say between the original image or 
the initial document long and the reference document 1^^,, and according to law of correspondence between GAP and 
noised marking image Ij^ ^ Particularty effective results rise from the use of the following law, described in 
detail in the application PCT/FR99/00485 and which contents are herein incorporated by reference : 

if (GAP(x,y) < 0 and ^ i(x,y) = 0) Or (GAP(x.y) > 0 and 1^^ i(x,y) = 1) 
5^ then Ij^ 2(x.y) = GAP(x,y) 

if not lj„, 2(x,y) = 0 which Is concisely written: 

tint 2(x. y) = In^ (GAP(x. y); 0) or Sup (GAP(x, y). 0) according to whether Ijn, , is respectively equal to 0 or to 1. 
[0114] Operators inf and sup respectively represent the mathematical operators providing respectively the 
smallest and the largest values considered. 

[0115] This criterion of correspondence between GAP and Ij^ ^ makes it possible to establish a binary link 
between GAP which is coded by its sign (positive or negative), and numerical information to hide from considered the 
pixel, which is worth 1 or 0. Indeed, it is clear that when Ij^i ,=0, only the values of GAP which are negative will be 
added to the reference document 1^^, and will carry information. Conversely, when Ij^^ ^-1. only positive values of GAP 
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are added to the reference document and are consequently carrying information. In all the other cases, the value 
of Ijn, 2 wi" ^® to 0, which will comes to returning the approximated value of the reference document 1^^ to build 
modified image 1*0^0. Rather curiously with the invention it has been seen that the pixels which will be «carrying » 
primary digital information will tend to qualitatively move away image from l^^f and, on the contrary, to bring it closer to I 
^ oriQ, thus increasing the original fidelity of the signed image towards the original image. Various alternatives can 
be introduced on the way of modulating Ijnf 2 between 0 and the value of GAP, by choosing other terminal values, for 
example 1 and 1 ,2 times the value of GAP, instead of 0 and one time the value of GAP, 

[0116] Once determined, internal mark Ijnf 2 can then be added to image Ir^f. which makes it possible to produce 
the signed image y^^^ to be broadcast. Consequently this signed image can be used, transmitted and undergo 
transformations. When at a given moment the transmitter of signed image (or any interested third party), is in 
10 presence of a similar image, he/she will want to check if this image V^^^ carries a signature, and which one. 

[0117] The extraction process of hidden information 2 ^^om a possibly modified image V^^^ will now be 
described. To this end, one refers to figure 8. In a dual way to insertion, the extraction will be carried out in 
the following way, as schematized in figure 2. Step 111 result to calculation of the auto-similarity W ' code 
associated with I'oria by using the same context as that used for the insertion operation. This results in a IFS code 
which allows, in step 112, building of a reference document, noted 1^^, in the general case, or A' in the event of 
fractal calculation of \\gf by means of a contractive IFS code. Support image I',, that is to say can then be recovered 
at step 113, by means of the follov\ring relation; 

' ifTf2 " ' orig * ^ 

[01 18] It should be noted that A' is to 1'^^^ what A was to ^0 in the phase of information insertion in 1^^^. The codes 
IFS which are associated with the Iq^o images and A are identical (the first by construction and the second by 
definition of an attractor) and, moreover, each point-image of I'ong is included in the interval defined by its value 
in long and A. 

[0119] Then, in step 114, primary digital information I'jn, ^ is determined from secondary digital information rint_2 by 
means of a test corresponding to the insertion operation : 

if I'in, 2(x,y) > predetermined threshold, then \\^_^ {x,y) = 1 

if y-mij^^y) < threshold then (x.y) = 0 
if not a one cannot say »(for the pixel (x,y) does not contain any hidden information). By default, the predetermined 
threshold is fixed to 0, but other larger thresholds can be used to adapt the robustness characteristics of 
algorithm to the user's needs : other «larger thresholds » can be used in order to limit the risks to be mistaken in 
each point (x,y). but in this case the number of binary data recovered from whole hidden information is similady 
decreased. 

[0120] Thus the result of the extraction phase of step 114 allows of product a I'l^f ^ image which is likely to 
contain, in a form possibly degraded by intermediate transformations of the signed image, logo ti^f ^ which was used 
during the signature. 

35 [0121] The process then continues with postprocessing steps 115 and 116 which enable rebuilding, if necessary, 
of the Ijn, ologo which was used during the signature, from its I'j^ ^ image, recovered in extraction steps 111-114. 
[0122] ' The document \\^ ^ corresponds to a binary noised and possibly degraded and unreadable representation 
of the original logo Ij^, q. s^er redundancy and formatting. It is necessary, as is illustrated in step 115, to proceed 
to a resynchronization between the local noise B regenerated from the secret key K,, and signature Tj^f ^ The 
technique of reframing the supposed signature I'jn, ^ and the noise locally introduced during the formation of signature I 
inf 1, is again based on a search for coherence between the various bits contributing to the rebuilding of a pixel of 
the primary logo Ij,,, To this end, a criterion built on the basis of two subcriteria related to the two levels of 
preprocessing of step 4 of figure 1 - namely the local enlargement of a factor K, and of total replication I times 
of enlarged logo Ij^f 0 maximized. 

[0123] The first local subcriterlon is related to oversampling of the logo and checks that the bit value is 
^5 constant over the period of oversampling. This condition is met only if the noise is correctiy reframed with I'jnf Indeed, 
before noising of signature li„f 0.2- the bit value of the signature is locally reproduced k times around this bit, and 
the value of this bit is still reproduced L times on the level of the total signature, because of the replication of 
the redundant signature I inf o- 2 before it was noised. 

[0124] Consequentiy, by eliminating noise influence on the supposed signature I'jnf ^ one must find the same type 
^ of local and total coherence. It is thus enough to test several possible transformations relating to I'jnf ^ and selected in 
a dictionary and, several types of relative reframing of the noise compared with the supposed signature Tj^f 1. and 
according to the results, one v^ll know if the image to be tested is signed and the then signature will 
automatically be extracted. Thus, as at step 16 of figure 2, reframing of the noise and signature I'lnf 1 in step 115 is 
done by a series iterations through which successive transformations are applied to I'jn, 1 and, for each one of them, 
its effect on coherence of value I'jnf 1 XOR B, that is to say I'jn, 0.2-. ® considered correct when both redundancy 
55 subcriteria are satisfied. 

[0125] Reframing of the noise with 1';^ , is illustrated in connection with figures 9a and 9b, where the 
successive states of I'jnf i, of the noise B, and the function I'jnf ^XOR B (represented in fig. 10 in reference vM\ process 
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step 120) are shown. In figure 9a. the bit value of I'jnf i XOR B is not constant over the period of oversampling (here 
three basic times), which means that the noise B is not correctly not reframed. In figure 9b, to. illustrate 
reframing effect, noise B has been offset by one point towards the right In this elementary example values of bits of r 
inf 1 XOR B have become constant on period of oversampling, and it can be concluded that noise B is now correctly 
reframed in relation to \'^„f which means there is local coherence, in the vicinity of the tested area of I'jn, 

^ (0126] In order to test this coherence, based on both preceding subcriterion at the local and total level, a 
function known as coherence function is built. In reference vyrtth figure 10, a score of coherence (step 121) to 
maximize is calculated. This score is calculated as follows : for each basic binary bit of the logo, on all its 
occurrences in I'ln, o.2' difference in absolute value between the pixels whose binary value is to zero with those 
whose binary value is to one is calculated, then all n pixels of the basic logo Ij^, q are added up, this sum constituting a 

iO score of coherence to maximize by use of possible transformations applied to the supposed signature. Reframing is 
then similar to the maximization of this function in relation to the variable of the transformations which takes its 
values in a predefined set : offset, scaling factor, rotation etc ... In practice, it is like iteratively testing 
the value of the function of coherence for a whole series of transformations T of the image, and retaining the 
transformation or the combination of transformations which maximizes the function of coherence. It should be noted 
that the calculation of each score is independent of the others, and all calculations can be made in parallel. In 

15 other terms, all transformations T to be tested, to each of which will correspond a score, could be realized in 
parallel. These transformations comprise all the operations of image translations, mirroring effect, of positive and 
negative zooming, or combinations thereof. The method could be immediately extended to other transformations if 
needed. 

[0127] Once reframing is complete, if necessary, the revealed redundancy can be exploited to regenerate the 
signature registered in image loria- This exploitation consists in reversing redundancy adding by replacing the kxl 
pixels which «vote » in a certain manner by a single pixel in order to pass from document 0.2 document I'j^ q. 
Step of process 116 thus results in organizing a «vote », possibly weighted, with information resulting from the 
reframing of step 115, concerning occurrences value (0, 1, unspecified) among all occurrences of this element in the 
primary formatted signature Tip, 0.2- ^^^^ 3 score higher than a certain predetermined 

threshold, the logo pixel of takes the value of the dominant group of pixels. The precise value of the threshold to 
25 set can be determined by people qualified in the art At the end of voting step 1 16, a binary image Tj^f q is obtained. If 
modified image I'onp was obtained from transformations of an image signed with signature li^, q, then t'jnf q will be 
identical to ij^f o> vvithin a few bits sometimes. Concretely, if the inserted signature Ij^f q is a visible logo (for example 
Eurecom's logo such as shown in figure 3), the recovered signature q ^ immediately recognized by means of a 
mere visual examination. 

[0128] Thus, as tt is noted, the invention enables automatic recovery of a signature which is written into a 
signed document, and further allows use of large signatures while reducing the distortion introduced into the image 
because of the signature writing. 

[0129] The insertion and extraction method were particuiariy described In relation to fractal analysis method, 
with or without contractive code, which produces good results, tt is however necessary to note that, in the 
particular case of digital Imagery, the process of the invention can be used v^h other approximation methods, for 
35 example a Jpeg coding, a cosine coding or a simple process of resolution reduction. 

Claims 

1. Method of marking a multi-media document 1^^, such as a fixed or animated image, by generating an external 
mark from said document and a marking image Ij^, 0. characterized in that the method comprises the following 
40 steps : 

decomposing (1) the Initial document lorig according to a fractal analysis method revealing a IFS code ; 

determining (2) a reference image 1,^, corresponding to said IFS code ; 

45 ' point-by-point subtracting (3) the initial document lorja and reference image I and according to least one 

vector representative of the image, then thresholding the result to produce a binary support image Ig ; 



formatting (4) the marking Image li^,^ so as to obtain an image of the same size as the support image 1, by pre- 
introducing at least one level of redundancy ; 

polnt-by-point combining (6) the marking image Ij^ q. formatted beforehand, with said support image by means 
of a logical operation, for example XOR or inverse XOR, so as to generate an external mark which can be filed. 



2. Method according to claim 1 characterized in that the IFS code is of contractive type and that image formatting 
is then followed by a noisying operation by means of a secret key Ks, 

3. Method according to claim 2 characterized in that the step of fomnatting the marking image Ij^f 0 includes : 
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a first step of duplication of each pixel of the marking image so as to introduce a vertical and horizontal 
redundancy, on a local level in a vicinity of each pixel ; : 

a second step of duplication on a global level consisting in reproducing several times the result of said 
^ first step. 

4. Method according to claim 1 characterized in that to determine a image of reference l^et. ^rom the initial document I 
orig . the fractal method is used, which consists in : 

- cutting out the initial document lo^g into a set of primitives range(i) covering the totality of the initial document; 

extracting a set of primitives domain(j) from the initial document I ^rig. 'f^ ^^^^^ to create a dictionary of 
basic primitives domain(j) (Dbase) ; 

determining an additional dictionary of primitive domain (Dfinal), from the dictionary 
15 of basic primitives domain (Dbase), by applying a set of local geometric and/or photometric transformations 

Wi to the basic primitives ; 

matching each basic primitive range(0 with an element of the dictionaryCDffna()» such that each 
primitive range(i) is equal to a primitive domainsO) transform by a local transformation wi, within about an error 
(epsilon), the image made up of all the primitives range(i) approximated from a 
20 primitive domainQ) constituting reference image I ^. 

5. Method of checking a multi-media document intended to bring relate it to a mark generated according to the 
method of claim 1 and stored in a database, characterized in that it comprises the following steps : 

- decomposing (1 1) the multi-media document according to a fractal analysis method providing a IFS code, 
in particular of contractive type; 

determining (12) an attractor image corresponding to said IFS code ; 

20 - point-by-point subtracting (13) the multimedia document X^^^ and attractor image r^ef, and according to at 

least one vector representative of the image, then thresholding the result to produce a binary support image l^ ; 

combining (16, 17) the disnoised mark by means of XOR or inverse XOR logical operation with said support 
image of \\ by preliminary proceeding to reframing of the latter to take into account any possible 
transformation undergone by said image to analyze. 

35 

formatting the result of the combination so as to excerpt the marking image written inside said mark. 

6. Method according to claim 5 characterized in that said reframing operation comprises the following steps : 

^ - point-by-point combining (16), the disnoised martc with said support image of \\ by means of a logical 

operation, by proceeding to a preliminary elementary geometric transformation T on said support image \\ ; 

proceeding to an iteration of combinations for a whole collection of elementary transformations defined in a 
dictionary and, for each one of them, calculating a score of coherence intended to maximize the redundancy 
^3 effect potentially existing on a local and also general level. 

7. Use of the method of generating a mark according to daim 1 in a certification server allowing automatic 
recording of multimedia documents, via a telecommunications network by means of a process comprising : 

50 - receiving (21 ) a recording request arriving to the certification server through the network ; 

transmitting (22) a request inviting the requesting user to produce his/her personal Identification data 
associated to a password, if necessary ; 



55 



establishing (23) a digital image recording transaction; 

receiving (24) a digital image produced by the requesting user, and generating a mark corresponding to said 
image, then filing said mark ; 
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transmitting (25) the filed identification data of said mark to the requesting user; 

continuing recording until the last image subjected by the requesting user has been recorded , and 

closing the recording session. 

Method according to claim 1 characterized in that the marking file Ijn, q comprises control information to be 
inserted into the external mark, and likely to be detected by a telecommunications network control system in 
order to intercept the multi-media document circulating within the network. 

Method of marking of a multi-media document lono. such as a fixed or animated Image, by generating a mark from 
said document and a marking Image Ijnf q, characterized in that the method comprises the following steps : 



decomposing (1) the initial document long according to a lossy approximation method; 

15 - determining (2) a reference image 1^^; 

point-by-point subtracting (3) the initial document 1^^^^ and reference image 1^0,, and according to at least one 
vector representative of the image in order to produce a support image I, ; 

formatting (4) the marking image lin, q so as to obtain an image of the same size as the support image Ij by 
pre-introducing at least one level of redundancy: 

point-by-point combining (6) the marking image Ij^f q, formatted beforehand, with the support image so as to 
generate a mark or a signature which can be, according to the case, either filed; or internally reintroduced 



into the initial document lo^g. 



10. Method according to claim 9 characterized in that the lossy approximation method is a autosimilarity method 
based fractal analysis, and in that the mark Is an internal mark I inf-2 which is reintroduced into image 1^^ and is 
point-by-point calculated according to the GAP value of support image 1^ and according to a law of correspondence 
between the GAP value and said formatted marking image. 

11. Method according to claim 10 characterized in that step of formatting marking image Ij^f o includes : 

a first step of duplication of each pixel of the marking image so as to introduce, on a local level in a 
vicinity of each pixel, a vertical and horizontal redundancy ; 

a second step of duplication on a total level, consisting in reproducing several times the result of said first step; 
a step of pseudo-random noising the signature in order to obtain a noised binary document Ijpf ^ 
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12. Method according to claim 1 1 characterized in that said internal mark Ij^f 2 is determined by the follov^^ng formula : 

linf.2 y) = inf (GAP(x, y); THRESHOLD) or sup (GAP(x. y), THRESHOLD), 
according to whether Ijnt i(x, y) is respectively equal to 0 or 1, or conversely, with THRESHOLD equal to a 
predetermined value; the latter being then combined with the reference document I ref by means of an OR exclusive 
(XOR) operation. 

13. Method of extracting a mark used as signature in a digital document Torifl to test, comprising the following steps : 
determining a digital reference document r^^ by means of a lossy approximation method ; 
determining the gap GAP between said document to test l'^^ and the reference document 
deducing signature 1';,^ ^ by comparing said GAP with a threshold value , 

reframing noise (b) and supposed signature I'in, lin order to maximize a function of coherence between the 
noise and the supposed signature; 

on reaching a maximum coherence, higher than a certain threshold of coherence, between noise (b) and the 
supposed signature 1. deducing from it that the noise and supposed signature are correctiy reframed, 
and that the image is a signed image. 
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14. Method according to claim 13 characterized in that noise (b) and the supposed signature I'in, i are OR exclusived 
to be reframed, in order to generate an image t'inf 0.2 then this image is checked for a degree of coherence, if it 
does not present any a transformation T is applied to the supposed signature iv^, and reframing process is 

^ repeated with other transformations T, until obtaining a maximum degree of coherence in image l-n, 0.2. 

15. Method according to claim 14, characterized in that to measure coherence of a supposed signature Ti^, for 
each basic binary bit of the logo, in all its occurrences, the difference in absolute value between the pixels 
whose binary value is to zero and those whose binary value is one Is calculated, then all n pixels of the basic logo I 

inf o afe added up, this sum constituting a score of coherence to maximize by use of possible transformations applied 
10 ~ to the supposed signature Tj^f ^. 

16. Method according to any of claims 13 to 15 characterized in that to reconstitute the primary signature Tj^f 0, for 
each primary signature element to be rebuilt, a vote about the occurrences value from (0, 1, unspecified) among 
all the occurrences of this element in the primary formatted signature Ij^, 0.2 organised, which can be balanced by 
information resulting from reframing. 
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17. Method according to the claim 9 or 13 characterized in that reference image 1^^ is determined by means of a lossy 
approximation method such as a Jpeg coding, a cosine coding or a coding by reduction of the image resolution. 
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